<!DOCTYPE html><html lang="zh-CN" data-theme="light"><head><meta charset="UTF-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><meta name="viewport" content="width=device-width,initial-scale=1"><title>零基础入门金融风控之贷款违约预测Task3：特征工程 | cattle_coder</title><meta name="author" content="cattle_coder,344285081@qq.com"><meta name="copyright" content="cattle_coder"><meta name="format-detection" content="telephone=no"><meta name="theme-color" content="#ffffff"><meta name="description" content="零基础入门金融风控之贷款违约预测Task3：特征工程特征工程是数据处理中最为重要的一部分，也是变化最多一部分。如果把特征工程给处理好，那么就等于成功一半。@[TOC] 导入需要的包 1234567891011121314151617import pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport seaborn a">
<meta property="og:type" content="article">
<meta property="og:title" content="零基础入门金融风控之贷款违约预测Task3：特征工程">
<meta property="og:url" content="https://azhangqingxiang.gitee.io/a_zhangqingxiang.gitee.io/2021/05/03/%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8BTask3%EF%BC%9A%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B/index.html">
<meta property="og:site_name" content="cattle_coder">
<meta property="og:description" content="零基础入门金融风控之贷款违约预测Task3：特征工程特征工程是数据处理中最为重要的一部分，也是变化最多一部分。如果把特征工程给处理好，那么就等于成功一半。@[TOC] 导入需要的包 1234567891011121314151617import pandas as pdimport numpy as npimport matplotlib.pyplot as pltimport seaborn a">
<meta property="og:locale" content="zh_CN">
<meta property="og:image" content="https://i.loli.net/2021/05/05/ZMyANcLThdbtsiC.jpg">
<meta property="article:published_time" content="2021-05-03T08:42:09.000Z">
<meta property="article:modified_time" content="2021-05-05T11:55:32.353Z">
<meta property="article:author" content="cattle_coder">
<meta name="twitter:card" content="summary">
<meta name="twitter:image" content="https://i.loli.net/2021/05/05/ZMyANcLThdbtsiC.jpg"><link rel="shortcut icon" href="/a_zhangqingxiang.gitee.io/img/favicon.png"><link rel="canonical" href="https://azhangqingxiang.gitee.io/a_zhangqingxiang.gitee.io/2021/05/03/%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8BTask3%EF%BC%9A%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B/"><link rel="preconnect" href="//cdn.jsdelivr.net"/><link rel="preconnect" href="//busuanzi.ibruce.info"/><meta name="yandex-verification" content="{&quot;enable&quot;:true,&quot;main&quot;:&quot;#49B1F5&quot;,&quot;paginator&quot;:&quot;#00c4b6&quot;,&quot;button_hover&quot;:&quot;#FF7242&quot;,&quot;text_selection&quot;:&quot;#00c4b6&quot;,&quot;link_color&quot;:&quot;#99a9bf&quot;,&quot;meta_color&quot;:&quot;#858585&quot;,&quot;hr_color&quot;:&quot;#A4D8FA&quot;,&quot;code_foreground&quot;:&quot;#F47466&quot;,&quot;code_background&quot;:&quot;rgba(27, 31, 35, .05)&quot;,&quot;toc_color&quot;:&quot;#00c4b6&quot;,&quot;blockquote_padding_color&quot;:&quot;#49b1f5&quot;,&quot;blockquote_background_color&quot;:&quot;#49b1f5&quot;}"/><link rel="stylesheet" href="/a_zhangqingxiang.gitee.io/css/index.css"><link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@fortawesome/fontawesome-free/css/all.min.css" media="print" onload="this.media='all'"><script>const GLOBAL_CONFIG = { 
  root: '/a_zhangqingxiang.gitee.io/',
  algolia: undefined,
  localSearch: undefined,
  translate: undefined,
  noticeOutdate: undefined,
  highlight: {"plugin":"highlighjs","highlightCopy":true,"highlightLang":true,"highlightHeightLimit":false},
  copy: {
    success: '复制成功',
    error: '复制错误',
    noSupport: '浏览器不支持'
  },
  relativeDate: {
    homepage: false,
    post: false
  },
  runtime: '',
  date_suffix: {
    just: '刚刚',
    min: '分钟前',
    hour: '小时前',
    day: '天前',
    month: '个月前'
  },
  copyright: {"limitCount":50,"languages":{"author":"作者: cattle_coder","link":"链接: ","source":"来源: cattle_coder","info":"著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。"}},
  lightbox: 'fancybox',
  Snackbar: undefined,
  source: {
    jQuery: 'https://cdn.jsdelivr.net/npm/jquery@latest/dist/jquery.min.js',
    justifiedGallery: {
      js: 'https://cdn.jsdelivr.net/npm/justifiedGallery/dist/js/jquery.justifiedGallery.min.js',
      css: 'https://cdn.jsdelivr.net/npm/justifiedGallery/dist/css/justifiedGallery.min.css'
    },
    fancybox: {
      js: 'https://cdn.jsdelivr.net/npm/@fancyapps/fancybox@latest/dist/jquery.fancybox.min.js',
      css: 'https://cdn.jsdelivr.net/npm/@fancyapps/fancybox@latest/dist/jquery.fancybox.min.css'
    }
  },
  isPhotoFigcaption: false,
  islazyload: true,
  isanchor: false
}</script><script id="config-diff">var GLOBAL_CONFIG_SITE = { 
  isPost: true,
  isHome: false,
  isHighlightShrink: false,
  isToc: true,
  postUpdate: '2021-05-05 19:55:32'
}</script><noscript><style type="text/css">
  #nav {
    opacity: 1
  }
  .justified-gallery img {
    opacity: 1
  }

  #recent-posts time,
  #post-meta time {
    display: inline !important
  }
</style></noscript><script>(win=>{
    win.saveToLocal = {
      set: function setWithExpiry(key, value, ttl) {
        if (ttl === 0) return
        const now = new Date()
        const expiryDay = ttl * 86400000
        const item = {
          value: value,
          expiry: now.getTime() + expiryDay,
        }
        localStorage.setItem(key, JSON.stringify(item))
      },

      get: function getWithExpiry(key) {
        const itemStr = localStorage.getItem(key)

        if (!itemStr) {
          return undefined
        }
        const item = JSON.parse(itemStr)
        const now = new Date()

        if (now.getTime() > item.expiry) {
          localStorage.removeItem(key)
          return undefined
        }
        return item.value
      }
    }
  
    win.getScript = url => new Promise((resolve, reject) => {
      const script = document.createElement('script')
      script.src = url
      script.async = true
      script.onerror = reject
      script.onload = script.onreadystatechange = function() {
        const loadState = this.readyState
        if (loadState && loadState !== 'loaded' && loadState !== 'complete') return
        script.onload = script.onreadystatechange = null
        resolve()
      }
      document.head.appendChild(script)
    })
  
      win.activateDarkMode = function () {
        document.documentElement.setAttribute('data-theme', 'dark')
        if (document.querySelector('meta[name="theme-color"]') !== null) {
          document.querySelector('meta[name="theme-color"]').setAttribute('content', '#0d0d0d')
        }
      }
      win.activateLightMode = function () {
        document.documentElement.setAttribute('data-theme', 'light')
        if (document.querySelector('meta[name="theme-color"]') !== null) {
          document.querySelector('meta[name="theme-color"]').setAttribute('content', '#ffffff')
        }
      }
      const t = saveToLocal.get('theme')
    
          if (t === 'dark') activateDarkMode()
          else if (t === 'light') activateLightMode()
        
      const asideStatus = saveToLocal.get('aside-status')
      if (asideStatus !== undefined) {
        if (asideStatus === 'hide') {
          document.documentElement.classList.add('hide-aside')
        } else {
          document.documentElement.classList.remove('hide-aside')
        }
      }
    })(window)</script><meta name="generator" content="Hexo 5.4.0"><link rel="alternate" href="/a_zhangqingxiang.gitee.io/atom.xml" title="cattle_coder" type="application/atom+xml">
</head><body><div id="sidebar"><div id="menu-mask"></div><div id="sidebar-menus"><div class="author-avatar"><img class="avatar-img" data-lazy-src="https://i.loli.net/2021/05/03/fjVLMHmWeun5wxX.jpg" onerror="onerror=null;src='/img/friend_404.gif'" alt="avatar"/></div><div class="site-data"><div class="data-item is-center"><div class="data-item-link"><a href="/a_zhangqingxiang.gitee.io/archives/"><div class="headline">文章</div><div class="length-num">15</div></a></div></div><div class="data-item is-center"><div class="data-item-link"><a href="/a_zhangqingxiang.gitee.io/tags/"><div class="headline">标签</div><div class="length-num">2</div></a></div></div><div class="data-item is-center"><div class="data-item-link"><a href="/a_zhangqingxiang.gitee.io/categories/"><div class="headline">分类</div><div class="length-num">6</div></a></div></div></div><hr/><div class="menus_items"><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/"><i class="fa-fw fas fa-home"></i><span> 首页</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/archives/"><i class="fa-fw fas fa-archive"></i><span> 时间轴</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/tags/"><i class="fa-fw fas fa-tags"></i><span> 标签</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/categories/"><i class="fa-fw fas fa-folder-open"></i><span> 分类</span></a></div><div class="menus_item"><a class="site-page" href="javascript:void(0);"><i class="fa-fw fa fa-heartbeat"></i><span> 清单</span><i class="fas fa-chevron-down expand"></i></a><ul class="menus_item_child"><li><a class="site-page child" href="/a_zhangqingxiang.gitee.io/music/"><i class="fa-fw fas fa-music"></i><span> 音乐</span></a></li></ul></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/link/"><i class="fa-fw fas fa-link"></i><span> 友链</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/about/"><i class="fa-fw fas fa-heart"></i><span> 关于</span></a></div></div></div></div><div class="post" id="body-wrap"><header class="post-bg" id="page-header" style="background-image: url('https://i.loli.net/2021/05/05/ZMyANcLThdbtsiC.jpg')"><nav id="nav"><span id="blog_name"><a id="site-name" href="/a_zhangqingxiang.gitee.io/">cattle_coder</a></span><div id="menus"><div class="menus_items"><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/"><i class="fa-fw fas fa-home"></i><span> 首页</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/archives/"><i class="fa-fw fas fa-archive"></i><span> 时间轴</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/tags/"><i class="fa-fw fas fa-tags"></i><span> 标签</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/categories/"><i class="fa-fw fas fa-folder-open"></i><span> 分类</span></a></div><div class="menus_item"><a class="site-page" href="javascript:void(0);"><i class="fa-fw fa fa-heartbeat"></i><span> 清单</span><i class="fas fa-chevron-down expand"></i></a><ul class="menus_item_child"><li><a class="site-page child" href="/a_zhangqingxiang.gitee.io/music/"><i class="fa-fw fas fa-music"></i><span> 音乐</span></a></li></ul></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/link/"><i class="fa-fw fas fa-link"></i><span> 友链</span></a></div><div class="menus_item"><a class="site-page" href="/a_zhangqingxiang.gitee.io/about/"><i class="fa-fw fas fa-heart"></i><span> 关于</span></a></div></div><div id="toggle-menu"><a class="site-page"><i class="fas fa-bars fa-fw"></i></a></div></div></nav><div id="post-info"><h1 class="post-title">零基础入门金融风控之贷款违约预测Task3：特征工程</h1><div id="post-meta"><div class="meta-firstline"><span class="post-meta-date"><i class="far fa-calendar-alt fa-fw post-meta-icon"></i><span class="post-meta-label">发表于</span><time class="post-meta-date-created" datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time><span class="post-meta-separator">|</span><i class="fas fa-history fa-fw post-meta-icon"></i><span class="post-meta-label">更新于</span><time class="post-meta-date-updated" datetime="2021-05-05T11:55:32.353Z" title="更新于 2021-05-05 19:55:32">2021-05-05</time></span><span class="post-meta-categories"><span class="post-meta-separator">|</span><i class="fas fa-inbox fa-fw post-meta-icon"></i><a class="post-meta-categories" href="/a_zhangqingxiang.gitee.io/categories/%E5%AD%A6%E4%B9%A0%E8%B5%9B/">学习赛</a></span></div><div class="meta-secondline"><span class="post-meta-separator">|</span><span class="post-meta-pv-cv" id="" data-flag-title="零基础入门金融风控之贷款违约预测Task3：特征工程"><i class="far fa-eye fa-fw post-meta-icon"></i><span class="post-meta-label">阅读量:</span><span id="busuanzi_value_page_pv"></span></span></div></div></div></header><main class="layout" id="content-inner"><div id="post"><article class="post-content" id="article-container"><h1 id="零基础入门金融风控之贷款违约预测"><a href="#零基础入门金融风控之贷款违约预测" class="headerlink" title="零基础入门金融风控之贷款违约预测"></a>零基础入门金融风控之贷款违约预测</h1><h1 id="Task3：特征工程"><a href="#Task3：特征工程" class="headerlink" title="Task3：特征工程"></a>Task3：特征工程</h1><p>特征工程是数据处理中最为重要的一部分，也是变化最多一部分。如果把特征工程给处理好，那么就等于成功一半。<br>@[TOC]</p>
<p>导入需要的包</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> pandas <span class="keyword">as</span> pd</span><br><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="keyword">import</span> seaborn <span class="keyword">as</span> sns</span><br><span class="line"><span class="keyword">import</span> datetime</span><br><span class="line"><span class="keyword">from</span> tqdm <span class="keyword">import</span> tqdm</span><br><span class="line"><span class="keyword">from</span> sklearn.preprocessing <span class="keyword">import</span> LabelEncoder</span><br><span class="line"><span class="keyword">from</span> sklearn.feature_selection <span class="keyword">import</span> SelectKBest</span><br><span class="line"><span class="keyword">from</span> sklearn.feature_selection <span class="keyword">import</span> chi2</span><br><span class="line"><span class="keyword">from</span> sklearn.preprocessing <span class="keyword">import</span> MinMaxScaler</span><br><span class="line"><span class="keyword">import</span> xgboost <span class="keyword">as</span> xgb</span><br><span class="line"><span class="keyword">import</span> lightgbm <span class="keyword">as</span> lgb</span><br><span class="line"><span class="keyword">from</span> catboost <span class="keyword">import</span> CatBoostRegressor</span><br><span class="line"><span class="keyword">import</span> warnings</span><br><span class="line"><span class="keyword">from</span> sklearn.model_selection <span class="keyword">import</span> StratifiedKFold, KFold</span><br><span class="line"><span class="keyword">from</span> sklearn.metrics <span class="keyword">import</span> accuracy_score, f1_score, roc_auc_score, log_loss</span><br><span class="line">warnings.filterwarnings(<span class="string">&#x27;ignore&#x27;</span>)</span><br></pre></td></tr></table></figure>

<p>读取文件</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">data_train =pd.read_csv(<span class="string">&#x27;./train.csv&#x27;</span>)</span><br><span class="line">data_test_a = pd.read_csv(<span class="string">&#x27;./testA.csv&#x27;</span>)</span><br></pre></td></tr></table></figure>
<h4 id="特征预处理"><a href="#特征预处理" class="headerlink" title="特征预处理"></a>特征预处理</h4><p>首先我们查找出数据中的对象特征和数值特征<br>对数据需要有一定了解</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">numerical_fea = <span class="built_in">list</span>(data_train.select_dtypes(exclude=[<span class="string">&#x27;object&#x27;</span>]).columns)</span><br><span class="line">category_fea = <span class="built_in">list</span>(<span class="built_in">filter</span>(<span class="keyword">lambda</span> x: x <span class="keyword">not</span> <span class="keyword">in</span> numerical_fea,<span class="built_in">list</span>(data_train.columns)))</span><br><span class="line">label = <span class="string">&#x27;isDefault&#x27;</span></span><br><span class="line">numerical_fea.remove(label)</span><br></pre></td></tr></table></figure>

<p>数据大概了解</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data_train.info()</span><br></pre></td></tr></table></figure>

<pre><code>&lt;class &#39;pandas.core.frame.DataFrame&#39;&gt;
RangeIndex: 800000 entries, 0 to 799999
Data columns (total 47 columns):
id                    800000 non-null int64
loanAmnt              800000 non-null float64
term                  800000 non-null int64
interestRate          800000 non-null float64
installment           800000 non-null float64
grade                 800000 non-null object
subGrade              800000 non-null object
employmentTitle       799999 non-null float64
employmentLength      753201 non-null object
homeOwnership         800000 non-null int64
annualIncome          800000 non-null float64
verificationStatus    800000 non-null int64
issueDate             800000 non-null object
isDefault             800000 non-null int64
purpose               800000 non-null int64
postCode              799999 non-null float64
regionCode            800000 non-null int64
dti                   799761 non-null float64
delinquency_2years    800000 non-null float64
ficoRangeLow          800000 non-null float64
ficoRangeHigh         800000 non-null float64
openAcc               800000 non-null float64
pubRec                800000 non-null float64
pubRecBankruptcies    799595 non-null float64
revolBal              800000 non-null float64
revolUtil             799469 non-null float64
totalAcc              800000 non-null float64
initialListStatus     800000 non-null int64
applicationType       800000 non-null int64
earliesCreditLine     800000 non-null object
title                 799999 non-null float64
policyCode            800000 non-null float64
n0                    759730 non-null float64
n1                    759730 non-null float64
n2                    759730 non-null float64
n2.1                  759730 non-null float64
n4                    766761 non-null float64
n5                    759730 non-null float64
n6                    759730 non-null float64
n7                    759730 non-null float64
n8                    759729 non-null float64
n9                    759730 non-null float64
n10                   766761 non-null float64
n11                   730248 non-null float64
n12                   759730 non-null float64
n13                   759730 non-null float64
n14                   759730 non-null float64
dtypes: float64(33), int64(9), object(5)
memory usage: 286.9+ MB
</code></pre>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#查看数值的特征</span></span><br><span class="line">numerical_fea</span><br></pre></td></tr></table></figure>




<pre><code>[&#39;id&#39;,
 &#39;loanAmnt&#39;,
 &#39;term&#39;,
 &#39;interestRate&#39;,
 &#39;installment&#39;,
 &#39;employmentTitle&#39;,
 &#39;homeOwnership&#39;,
 &#39;annualIncome&#39;,
 &#39;verificationStatus&#39;,
 &#39;purpose&#39;,
 &#39;postCode&#39;,
 &#39;regionCode&#39;,
 &#39;dti&#39;,
 &#39;delinquency_2years&#39;,
 &#39;ficoRangeLow&#39;,
 &#39;ficoRangeHigh&#39;,
 &#39;openAcc&#39;,
 &#39;pubRec&#39;,
 &#39;pubRecBankruptcies&#39;,
 &#39;revolBal&#39;,
 &#39;revolUtil&#39;,
 &#39;totalAcc&#39;,
 &#39;initialListStatus&#39;,
 &#39;applicationType&#39;,
 &#39;title&#39;,
 &#39;policyCode&#39;,
 &#39;n0&#39;,
 &#39;n1&#39;,
 &#39;n2&#39;,
 &#39;n2.1&#39;,
 &#39;n4&#39;,
 &#39;n5&#39;,
 &#39;n6&#39;,
 &#39;n7&#39;,
 &#39;n8&#39;,
 &#39;n9&#39;,
 &#39;n10&#39;,
 &#39;n11&#39;,
 &#39;n12&#39;,
 &#39;n13&#39;,
 &#39;n14&#39;]
</code></pre>
<h4 id="缺失值处理"><a href="#缺失值处理" class="headerlink" title="缺失值处理"></a>缺失值处理</h4><ul>
<li>把所有缺失值替换成指定值</li>
<li>用上面值填充ffill</li>
<li>用下面值填充bfill<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#查看缺失值情况</span></span><br><span class="line">data_train.isnull().<span class="built_in">sum</span>()</span><br></pre></td></tr></table></figure></li>
</ul>
<pre><code>id                        0
loanAmnt                  0
term                      0
interestRate              0
installment               0
grade                     0
subGrade                  0
employmentTitle           1
employmentLength      46799
homeOwnership             0
annualIncome              0
verificationStatus        0
issueDate                 0
isDefault                 0
purpose                   0
postCode                  1
regionCode                0
dti                     239
delinquency_2years        0
ficoRangeLow              0
ficoRangeHigh             0
openAcc                   0
pubRec                    0
pubRecBankruptcies      405
revolBal                  0
revolUtil               531
totalAcc                  0
initialListStatus         0
applicationType           0
earliesCreditLine         0
title                     1
policyCode                0
n0                    40270
n1                    40270
n2                    40270
n2.1                  40270
n4                    33239
n5                    40270
n6                    40270
n7                    40270
n8                    40271
n9                    40270
n10                   33239
n11                   69752
n12                   40270
n13                   40270
n14                   40270
dtype: int64
</code></pre>
<p>这里我们可以平均值填充也可以众数。注意mode方法。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#这里的mode方法是去每一列的众数</span></span><br><span class="line">data_train[category_fea].mode()</span><br></pre></td></tr></table></figure>


<div>
<style scoped>
    .dataframe tbody tr th:only-of-type {
        vertical-align: middle;
    }

<pre><code>.dataframe tbody tr th &#123;
    vertical-align: top;
&#125;

.dataframe thead th &#123;
    text-align: right;
&#125;
</code></pre>
<p></style></p>
<table border="1" class="dataframe">
  <thead>
    <tr style="text-align: right;">
      <th></th>
      <th>grade</th>
      <th>subGrade</th>
      <th>employmentLength</th>
      <th>issueDate</th>
      <th>earliesCreditLine</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>0</td>
      <td>B</td>
      <td>C1</td>
      <td>10+ years</td>
      <td>2016-03-01</td>
      <td>Aug-2001</td>
    </tr>
  </tbody>
</table>
</div>




<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#按照平均数填充数值型特征</span></span><br><span class="line">data_train[numerical_fea] = data_train[numerical_fea].fillna(data_train[numerical_fea].median())</span><br><span class="line">data_test_a[numerical_fea] = data_test_a[numerical_fea].fillna(data_train[numerical_fea].median())</span><br><span class="line"><span class="comment">#按照众数填充类别型特征</span></span><br><span class="line">data_train[category_fea] = data_train[category_fea].fillna(data_train[category_fea].mode())</span><br><span class="line">data_test_a[category_fea] = data_test_a[category_fea].fillna(data_train[category_fea].mode())</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data_train.isnull().<span class="built_in">sum</span>()</span><br></pre></td></tr></table></figure>




<pre><code>id                        0
loanAmnt                  0
term                      0
interestRate              0
installment               0
grade                     0
subGrade                  0
employmentTitle           0
employmentLength      46799
homeOwnership             0
annualIncome              0
verificationStatus        0
issueDate                 0
isDefault                 0
purpose                   0
postCode                  0
regionCode                0
dti                       0
delinquency_2years        0
ficoRangeLow              0
ficoRangeHigh             0
openAcc                   0
pubRec                    0
pubRecBankruptcies        0
revolBal                  0
revolUtil                 0
totalAcc                  0
initialListStatus         0
applicationType           0
earliesCreditLine         0
title                     0
policyCode                0
n0                        0
n1                        0
n2                        0
n2.1                      0
n4                        0
n5                        0
n6                        0
n7                        0
n8                        0
n9                        0
n10                       0
n11                       0
n12                       0
n13                       0
n14                       0
dtype: int64
</code></pre>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#查看类别特征</span></span><br><span class="line">category_fea</span><br></pre></td></tr></table></figure>




<pre><code>[&#39;grade&#39;, &#39;subGrade&#39;, &#39;employmentLength&#39;, &#39;issueDate&#39;, &#39;earliesCreditLine&#39;]
</code></pre>
<h4 id="时间格式处理"><a href="#时间格式处理" class="headerlink" title="时间格式处理"></a>时间格式处理</h4><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#转化成时间格式</span></span><br><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data[<span class="string">&#x27;issueDate&#x27;</span>] = pd.to_datetime(data[<span class="string">&#x27;issueDate&#x27;</span>],<span class="built_in">format</span>=<span class="string">&#x27;%Y-%m-%d&#x27;</span>)</span><br><span class="line">    startdate = datetime.datetime.strptime(<span class="string">&#x27;2007-06-01&#x27;</span>, <span class="string">&#x27;%Y-%m-%d&#x27;</span>)</span><br><span class="line">    <span class="comment">#构造时间特征</span></span><br><span class="line">    data[<span class="string">&#x27;issueDateDT&#x27;</span>] = data[<span class="string">&#x27;issueDate&#x27;</span>].apply(<span class="keyword">lambda</span> x: x-startdate).dt.days</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data_train[<span class="string">&#x27;employmentLength&#x27;</span>].value_counts(dropna=<span class="literal">False</span>).sort_index()</span><br></pre></td></tr></table></figure>




<pre><code>1 year        52489
10+ years    262753
2 years       72358
3 years       64152
4 years       47985
5 years       50102
6 years       37254
7 years       35407
8 years       36192
9 years       30272
&lt; 1 year      64237
NaN           46799
Name: employmentLength, dtype: int64
</code></pre>
<h4 id="对象特征转化数值特征"><a href="#对象特征转化数值特征" class="headerlink" title="对象特征转化数值特征"></a>对象特征转化数值特征</h4><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">employmentLength_to_int</span>(<span class="params">s</span>):</span></span><br><span class="line">    <span class="keyword">if</span> pd.isnull(s):</span><br><span class="line">        <span class="keyword">return</span> s</span><br><span class="line">    <span class="keyword">else</span>:</span><br><span class="line">        <span class="keyword">return</span> np.int8(s.split()[<span class="number">0</span>])</span><br><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data[<span class="string">&#x27;employmentLength&#x27;</span>].replace(to_replace=<span class="string">&#x27;10+ years&#x27;</span>, value=<span class="string">&#x27;10 years&#x27;</span>, inplace=<span class="literal">True</span>)</span><br><span class="line">    data[<span class="string">&#x27;employmentLength&#x27;</span>].replace(<span class="string">&#x27;&lt; 1 year&#x27;</span>, <span class="string">&#x27;0 years&#x27;</span>, inplace=<span class="literal">True</span>)</span><br><span class="line">    data[<span class="string">&#x27;employmentLength&#x27;</span>] = data[<span class="string">&#x27;employmentLength&#x27;</span>].apply(employmentLength_to_int)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data[<span class="string">&#x27;employmentLength&#x27;</span>].value_counts(dropna=<span class="literal">False</span>).sort_index()</span><br></pre></td></tr></table></figure>




<pre><code>0.0     15989
1.0     13182
2.0     18207
3.0     16011
4.0     11833
5.0     12543
6.0      9328
7.0      8823
8.0      8976
9.0      7594
10.0    65772
NaN     11742
Name: employmentLength, dtype: int64
</code></pre>
<p>sample方法随机取这一列的5个值</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data_train[<span class="string">&#x27;earliesCreditLine&#x27;</span>].sample(<span class="number">5</span>)</span><br></pre></td></tr></table></figure>




<pre><code>250886    Nov-2000
528953    Sep-2000
703638    Jun-1989
80494     Mar-2009
715192    Dec-2003
Name: earliesCreditLine, dtype: object
</code></pre>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data[<span class="string">&#x27;earliesCreditLine&#x27;</span>] = data[<span class="string">&#x27;earliesCreditLine&#x27;</span>].apply(<span class="keyword">lambda</span> s: <span class="built_in">int</span>(s[-<span class="number">4</span>:]))</span><br></pre></td></tr></table></figure>

<h4 id="类别特征处理"><a href="#类别特征处理" class="headerlink" title="类别特征处理"></a>类别特征处理</h4><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 部分类别特征</span></span><br><span class="line">cate_features = [<span class="string">&#x27;grade&#x27;</span>, <span class="string">&#x27;subGrade&#x27;</span>, <span class="string">&#x27;employmentTitle&#x27;</span>, <span class="string">&#x27;homeOwnership&#x27;</span>, <span class="string">&#x27;verificationStatus&#x27;</span>, <span class="string">&#x27;purpose&#x27;</span>, <span class="string">&#x27;postCode&#x27;</span>, <span class="string">&#x27;regionCode&#x27;</span>, \</span><br><span class="line">                 <span class="string">&#x27;applicationType&#x27;</span>, <span class="string">&#x27;initialListStatus&#x27;</span>, <span class="string">&#x27;title&#x27;</span>, <span class="string">&#x27;policyCode&#x27;</span>]</span><br><span class="line"><span class="keyword">for</span> f <span class="keyword">in</span> cate_features:</span><br><span class="line">    <span class="built_in">print</span>(f, <span class="string">&#x27;类型数：&#x27;</span>, data[f].nunique())</span><br></pre></td></tr></table></figure>

<pre><code>grade 类型数： 7
subGrade 类型数： 35
employmentTitle 类型数： 79282
homeOwnership 类型数： 6
verificationStatus 类型数： 3
purpose 类型数： 14
postCode 类型数： 889
regionCode 类型数： 51
applicationType 类型数： 2
initialListStatus 类型数： 2
title 类型数： 12058
policyCod，e 类型数： 1
</code></pre>
<p>等级这种固定的类别特征可以使用匿名函数或者map去换比较方便。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data[<span class="string">&#x27;grade&#x27;</span>] = data[<span class="string">&#x27;grade&#x27;</span>].<span class="built_in">map</span>(&#123;<span class="string">&#x27;A&#x27;</span>:<span class="number">1</span>,<span class="string">&#x27;B&#x27;</span>:<span class="number">2</span>,<span class="string">&#x27;C&#x27;</span>:<span class="number">3</span>,<span class="string">&#x27;D&#x27;</span>:<span class="number">4</span>,<span class="string">&#x27;E&#x27;</span>:<span class="number">5</span>,<span class="string">&#x27;F&#x27;</span>:<span class="number">6</span>,<span class="string">&#x27;G&#x27;</span>:<span class="number">7</span>&#125;)</span><br></pre></td></tr></table></figure>
<p>类型数在2之上，又不是高维稀疏的,且纯分类特征</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 类型数在2之上，又不是高维稀疏的,且纯分类特征</span></span><br><span class="line"><span class="comment">#get_dummies这个方法可以构建矩阵</span></span><br><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data = pd.get_dummies(data, columns=[<span class="string">&#x27;subGrade&#x27;</span>, <span class="string">&#x27;homeOwnership&#x27;</span>, <span class="string">&#x27;verificationStatus&#x27;</span>, <span class="string">&#x27;purpose&#x27;</span>, <span class="string">&#x27;regionCode&#x27;</span>], drop_first=<span class="literal">True</span>)</span><br></pre></td></tr></table></figure>
<h4 id="异常值处理"><a href="#异常值处理" class="headerlink" title="异常值处理"></a>异常值处理</h4><ul>
<li>当我们在发现异常值得时候，一定要分清楚是什么原因导致异常值的。如果<strong>这异常值并不代表一种规律性</strong>话，是<strong>偶然的现象</strong>，或者说你并不想研究这种偶然的现象，这时可以将其<strong>删除</strong>。<strong>如果异常值存在且代表了一种真实存在的现象</strong>，那就不能随便删除。在现有的欺诈场景中很多时候欺诈数据本身相对于正常数据勒说就是异常的，我们要把这些异常点纳入，重新拟合模型，研究其规律。能用监督的用监督模型，不能用的还可以考虑用异常检测的算法来做。</li>
<li><strong>注意test的数据不能删</strong></li>
</ul>
<h5 id="对于检查异常方法之一：均方差"><a href="#对于检查异常方法之一：均方差" class="headerlink" title="对于检查异常方法之一：均方差"></a>对于检查异常方法之一：均方差</h5><p>基本上一些连续的数据都符合正态分布的</p>
<h5 id="方法二：箱型图"><a href="#方法二：箱型图" class="headerlink" title="方法二：箱型图"></a>方法二：箱型图</h5><p>将数据分成几个部分</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">find_outliers_by_3segama</span>(<span class="params">data,fea</span>):</span></span><br><span class="line">    data_std = np.std(data[fea])</span><br><span class="line">    data_mean = np.mean(data[fea])</span><br><span class="line">    outliers_cut_off = data_std * <span class="number">3</span></span><br><span class="line">    lower_rule = data_mean - outliers_cut_off</span><br><span class="line">    upper_rule = data_mean + outliers_cut_off</span><br><span class="line">    data[fea+<span class="string">&#x27;_outliers&#x27;</span>] = data[fea].apply(<span class="keyword">lambda</span> x:<span class="built_in">str</span>(<span class="string">&#x27;异常值&#x27;</span>) <span class="keyword">if</span> x &gt; upper_rule <span class="keyword">or</span> x &lt; lower_rule <span class="keyword">else</span> <span class="string">&#x27;正常值&#x27;</span>)</span><br><span class="line">    <span class="keyword">return</span> data</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">data_train = data_train.copy()</span><br><span class="line"><span class="keyword">for</span> fea <span class="keyword">in</span> numerical_fea:</span><br><span class="line">    data_train = find_outliers_by_3segama(data_train,fea)</span><br><span class="line">    <span class="built_in">print</span>(data_train[fea+<span class="string">&#x27;_outliers&#x27;</span>].value_counts())</span><br><span class="line">    <span class="built_in">print</span>(data_train.groupby(fea+<span class="string">&#x27;_outliers&#x27;</span>)[<span class="string">&#x27;isDefault&#x27;</span>].<span class="built_in">sum</span>())</span><br><span class="line">    <span class="built_in">print</span>(<span class="string">&#x27;*&#x27;</span>*<span class="number">10</span>)</span><br></pre></td></tr></table></figure>

<pre><code>正常值    800000
Name: id_outliers, dtype: int64
id_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    800000
Name: loanAmnt_outliers, dtype: int64
loanAmnt_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    800000
Name: term_outliers, dtype: int64
term_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    794259
异常值      5741
Name: interestRate_outliers, dtype: int64
interestRate_outliers
异常值      2916
正常值    156694
Name: isDefault, dtype: int64
**********
正常值    792046
异常值      7954
Name: installment_outliers, dtype: int64
installment_outliers
异常值      2152
正常值    157458
Name: isDefault, dtype: int64
**********
正常值    800000
Name: employmentTitle_outliers, dtype: int64
employmentTitle_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    799701
异常值       299
Name: homeOwnership_outliers, dtype: int64
homeOwnership_outliers
异常值        62
正常值    159548
Name: isDefault, dtype: int64
**********
正常值    793973
异常值      6027
Name: annualIncome_outliers, dtype: int64
annualIncome_outliers
异常值       756
正常值    158854
Name: isDefault, dtype: int64
**********
正常值    800000
Name: verificationStatus_outliers, dtype: int64
verificationStatus_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    783003
异常值     16997
Name: purpose_outliers, dtype: int64
purpose_outliers
异常值      3635
正常值    155975
Name: isDefault, dtype: int64
**********
正常值    798931
异常值      1069
Name: postCode_outliers, dtype: int64
postCode_outliers
异常值       221
正常值    159389
Name: isDefault, dtype: int64
**********
正常值    799994
异常值         6
Name: regionCode_outliers, dtype: int64
regionCode_outliers
异常值         1
正常值    159609
Name: isDefault, dtype: int64
**********
正常值    798440
异常值      1560
Name: dti_outliers, dtype: int64
dti_outliers
异常值       466
正常值    159144
Name: isDefault, dtype: int64
**********
正常值    778245
异常值     21755
Name: delinquency_2years_outliers, dtype: int64
delinquency_2years_outliers
异常值      5089
正常值    154521
Name: isDefault, dtype: int64
**********
正常值    788261
异常值     11739
Name: ficoRangeLow_outliers, dtype: int64
ficoRangeLow_outliers
异常值       778
正常值    158832
Name: isDefault, dtype: int64
**********
正常值    788261
异常值     11739
Name: ficoRangeHigh_outliers, dtype: int64
ficoRangeHigh_outliers
异常值       778
正常值    158832
Name: isDefault, dtype: int64
**********
正常值    790889
异常值      9111
Name: openAcc_outliers, dtype: int64
openAcc_outliers
异常值      2195
正常值    157415
Name: isDefault, dtype: int64
**********
正常值    792471
异常值      7529
Name: pubRec_outliers, dtype: int64
pubRec_outliers
异常值      1701
正常值    157909
Name: isDefault, dtype: int64
**********
正常值    794120
异常值      5880
Name: pubRecBankruptcies_outliers, dtype: int64
pubRecBankruptcies_outliers
异常值      1423
正常值    158187
Name: isDefault, dtype: int64
**********
正常值    790001
异常值      9999
Name: revolBal_outliers, dtype: int64
revolBal_outliers
异常值      1359
正常值    158251
Name: isDefault, dtype: int64
**********
正常值    799948
异常值        52
Name: revolUtil_outliers, dtype: int64
revolUtil_outliers
异常值        23
正常值    159587
Name: isDefault, dtype: int64
**********
正常值    791663
异常值      8337
Name: totalAcc_outliers, dtype: int64
totalAcc_outliers
异常值      1668
正常值    157942
Name: isDefault, dtype: int64
**********
正常值    800000
Name: initialListStatus_outliers, dtype: int64
initialListStatus_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    784586
异常值     15414
Name: applicationType_outliers, dtype: int64
applicationType_outliers
异常值      3875
正常值    155735
Name: isDefault, dtype: int64
**********
正常值    775134
异常值     24866
Name: title_outliers, dtype: int64
title_outliers
异常值      3900
正常值    155710
Name: isDefault, dtype: int64
**********
正常值    800000
Name: policyCode_outliers, dtype: int64
policyCode_outliers
正常值    159610
Name: isDefault, dtype: int64
**********
正常值    782773
异常值     17227
Name: n0_outliers, dtype: int64
n0_outliers
异常值      3485
正常值    156125
Name: isDefault, dtype: int64
**********
正常值    790500
异常值      9500
Name: n1_outliers, dtype: int64
n1_outliers
异常值      2491
正常值    157119
Name: isDefault, dtype: int64
**********
正常值    789067
异常值     10933
Name: n2_outliers, dtype: int64
n2_outliers
异常值      3205
正常值    156405
Name: isDefault, dtype: int64
**********
正常值    789067
异常值     10933
Name: n2.1_outliers, dtype: int64
n2.1_outliers
异常值      3205
正常值    156405
Name: isDefault, dtype: int64
**********
正常值    788660
异常值     11340
Name: n4_outliers, dtype: int64
n4_outliers
异常值      2476
正常值    157134
Name: isDefault, dtype: int64
**********
正常值    790355
异常值      9645
Name: n5_outliers, dtype: int64
n5_outliers
异常值      1858
正常值    157752
Name: isDefault, dtype: int64
**********
正常值    786006
异常值     13994
Name: n6_outliers, dtype: int64
n6_outliers
异常值      3182
正常值    156428
Name: isDefault, dtype: int64
**********
正常值    788430
异常值     11570
Name: n7_outliers, dtype: int64
n7_outliers
异常值      2746
正常值    156864
Name: isDefault, dtype: int64
**********
正常值    789625
异常值     10375
Name: n8_outliers, dtype: int64
n8_outliers
异常值      2131
正常值    157479
Name: isDefault, dtype: int64
**********
正常值    786384
异常值     13616
Name: n9_outliers, dtype: int64
n9_outliers
异常值      3953
正常值    155657
Name: isDefault, dtype: int64
**********
正常值    788979
异常值     11021
Name: n10_outliers, dtype: int64
n10_outliers
异常值      2639
正常值    156971
Name: isDefault, dtype: int64
**********
正常值    799434
异常值       566
Name: n11_outliers, dtype: int64
n11_outliers
异常值       112
正常值    159498
Name: isDefault, dtype: int64
**********
正常值    797585
异常值      2415
Name: n12_outliers, dtype: int64
n12_outliers
异常值       545
正常值    159065
Name: isDefault, dtype: int64
**********
正常值    788907
异常值     11093
Name: n13_outliers, dtype: int64
n13_outliers
异常值      2482
正常值    157128
Name: isDefault, dtype: int64
**********
正常值    788884
异常值     11116
Name: n14_outliers, dtype: int64
n14_outliers
异常值      3364
正常值    156246
Name: isDefault, dtype: int64
**********
</code></pre>
<p>删除异常值</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#删除异常值</span></span><br><span class="line"><span class="keyword">for</span> fea <span class="keyword">in</span> numerical_fea:</span><br><span class="line">    data_train = data_train[data_train[fea+<span class="string">&#x27;_outliers&#x27;</span>]==<span class="string">&#x27;正常值&#x27;</span>]</span><br><span class="line">    data_train = data_train.reset_index(drop=<span class="literal">True</span>) </span><br></pre></td></tr></table></figure>

<p>通过除法映射到间隔均匀的分箱中</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 通过除法映射到间隔均匀的分箱中，每个分箱的取值范围都是loanAmnt/1000</span></span><br><span class="line">data[<span class="string">&#x27;loanAmnt_bin1&#x27;</span>] = np.floor_divide(data[<span class="string">&#x27;loanAmnt&#x27;</span>], <span class="number">1000</span>)</span><br></pre></td></tr></table></figure>

<p>通过对数函数映射到指数宽度分箱</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">## 通过对数函数映射到指数宽度分箱</span></span><br><span class="line">data[<span class="string">&#x27;loanAmnt_bin2&#x27;</span>] = np.floor(np.log10(data[<span class="string">&#x27;loanAmnt&#x27;</span>]))</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">data[<span class="string">&#x27;loanAmnt_bin3&#x27;</span>] = pd.qcut(data[<span class="string">&#x27;loanAmnt&#x27;</span>], <span class="number">10</span>, labels=<span class="literal">False</span>)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">for</span> col <span class="keyword">in</span> [<span class="string">&#x27;grade&#x27;</span>, <span class="string">&#x27;subGrade&#x27;</span>]: </span><br><span class="line">    temp_dict = data_train.groupby([col])[<span class="string">&#x27;isDefault&#x27;</span>].agg([<span class="string">&#x27;mean&#x27;</span>]).reset_index().rename(columns=&#123;<span class="string">&#x27;mean&#x27;</span>: col + <span class="string">&#x27;_target_mean&#x27;</span>&#125;)</span><br><span class="line">    temp_dict.index = temp_dict[col].values</span><br><span class="line">    temp_dict = temp_dict[col + <span class="string">&#x27;_target_mean&#x27;</span>].to_dict()</span><br><span class="line"></span><br><span class="line">    data_train[col + <span class="string">&#x27;_target_mean&#x27;</span>] = data_train[col].<span class="built_in">map</span>(temp_dict)</span><br><span class="line">    data_test_a[col + <span class="string">&#x27;_target_mean&#x27;</span>] = data_test_a[col].<span class="built_in">map</span>(temp_dict)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 其他衍生变量 mean 和 std</span></span><br><span class="line"><span class="keyword">for</span> df <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    <span class="keyword">for</span> item <span class="keyword">in</span> [<span class="string">&#x27;n0&#x27;</span>,<span class="string">&#x27;n1&#x27;</span>,<span class="string">&#x27;n2&#x27;</span>,<span class="string">&#x27;n2.1&#x27;</span>,<span class="string">&#x27;n4&#x27;</span>,<span class="string">&#x27;n5&#x27;</span>,<span class="string">&#x27;n6&#x27;</span>,<span class="string">&#x27;n7&#x27;</span>,<span class="string">&#x27;n8&#x27;</span>,<span class="string">&#x27;n9&#x27;</span>,<span class="string">&#x27;n10&#x27;</span>,<span class="string">&#x27;n11&#x27;</span>,<span class="string">&#x27;n12&#x27;</span>,<span class="string">&#x27;n13&#x27;</span>,<span class="string">&#x27;n14&#x27;</span>]:</span><br><span class="line">        df[<span class="string">&#x27;grade_to_mean_&#x27;</span> + item] = df[<span class="string">&#x27;grade&#x27;</span>] / df.groupby([item])[<span class="string">&#x27;grade&#x27;</span>].transform(<span class="string">&#x27;mean&#x27;</span>)</span><br><span class="line">        df[<span class="string">&#x27;grade_to_std_&#x27;</span> + item] = df[<span class="string">&#x27;grade&#x27;</span>] / df.groupby([item])[<span class="string">&#x27;grade&#x27;</span>].transform(<span class="string">&#x27;std&#x27;</span>)</span><br></pre></td></tr></table></figure>

<h4 id="特征交互"><a href="#特征交互" class="headerlink" title="特征交互"></a>特征交互</h4><p>想要丰富特征，特别是对于线性模型而言，除了分箱外，另一种方法是添加原始数据的交互特征和多项式特征。</p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#label-encode:subGrade,postCode,title</span></span><br><span class="line"><span class="comment"># 高维类别特征需要进行转换</span></span><br><span class="line"><span class="keyword">for</span> col <span class="keyword">in</span> tqdm([<span class="string">&#x27;employmentTitle&#x27;</span>, <span class="string">&#x27;postCode&#x27;</span>, <span class="string">&#x27;title&#x27;</span>,<span class="string">&#x27;subGrade&#x27;</span>]):</span><br><span class="line">    le = LabelEncoder()</span><br><span class="line">    le.fit(<span class="built_in">list</span>(data_train[col].astype(<span class="built_in">str</span>).values) + <span class="built_in">list</span>(data_test_a[col].astype(<span class="built_in">str</span>).values))</span><br><span class="line">    data_train[col] = le.transform(<span class="built_in">list</span>(data_train[col].astype(<span class="built_in">str</span>).values))</span><br><span class="line">    data_test_a[col] = le.transform(<span class="built_in">list</span>(data_test_a[col].astype(<span class="built_in">str</span>).values))</span><br><span class="line"><span class="built_in">print</span>(<span class="string">&#x27;Label Encoding 完成&#x27;</span>)</span><br></pre></td></tr></table></figure>

<pre><code>100%|████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:12&lt;00:00,  3.15s/it]


Label Encoding 完成
</code></pre>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 删除不需要的数据</span></span><br><span class="line"><span class="keyword">for</span> data <span class="keyword">in</span> [data_train, data_test_a]:</span><br><span class="line">    data.drop([<span class="string">&#x27;issueDate&#x27;</span>], axis=<span class="number">1</span>,inplace=<span class="literal">True</span>)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="string">&quot;纵向用缺失值上面的值替换缺失值&quot;</span></span><br><span class="line">data_train = data_train.fillna(axis=<span class="number">0</span>,method=<span class="string">&#x27;ffill&#x27;</span>)</span><br><span class="line">x_train = data_train.drop([<span class="string">&#x27;isDefault&#x27;</span>,<span class="string">&#x27;id&#x27;</span>], axis=<span class="number">1</span>)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#计算协方差</span></span><br><span class="line">data_corr = x_train.corrwith(data_train.isDefault) <span class="comment">#计算相关性</span></span><br><span class="line">result = pd.DataFrame(columns=[<span class="string">&#x27;features&#x27;</span>, <span class="string">&#x27;corr&#x27;</span>])</span><br><span class="line">result[<span class="string">&#x27;features&#x27;</span>] = data_corr.index</span><br><span class="line">result[<span class="string">&#x27;corr&#x27;</span>] = data_corr.values</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 也可以画图图</span></span><br><span class="line">data_numeric = data_train[numerical_fea]</span><br><span class="line">correlation = data_numeric.corr()</span><br><span class="line"></span><br><span class="line">f , ax = plt.subplots(figsize = (<span class="number">7</span>, <span class="number">7</span>))</span><br><span class="line">plt.title(<span class="string">&#x27;Correlation of Numeric Features with Price&#x27;</span>,y=<span class="number">1</span>,size=<span class="number">16</span>)</span><br><span class="line">sns.heatmap(correlation,square = <span class="literal">True</span>,  vmax=<span class="number">0.8</span>)</span><br></pre></td></tr></table></figure>




<pre><code>&lt;matplotlib.axes._subplots.AxesSubplot at 0x23837a36f88&gt;
</code></pre>
<p><img src= "/a_zhangqingxiang.gitee.io/img/loading.gif" data-lazy-src="https://img-blog.csdnimg.cn/20200921220247630.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NTAxNDYzNA==,size_16,color_FFFFFF,t_70#pic_center" alt="在这里插入图片描述"></p>
<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">features = [f <span class="keyword">for</span> f <span class="keyword">in</span> data_train.columns <span class="keyword">if</span> f <span class="keyword">not</span> <span class="keyword">in</span> [<span class="string">&#x27;id&#x27;</span>,<span class="string">&#x27;issueDate&#x27;</span>,<span class="string">&#x27;isDefault&#x27;</span>] <span class="keyword">and</span> <span class="string">&#x27;_outliers&#x27;</span> <span class="keyword">not</span> <span class="keyword">in</span> f]</span><br><span class="line">x_train = data_train[features]</span><br><span class="line">x_test = data_test_a[features]</span><br><span class="line">y_train = data_train[<span class="string">&#x27;isDefault&#x27;</span>]</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">cv_model</span>(<span class="params">clf, train_x, train_y, test_x, clf_name</span>):</span></span><br><span class="line">    folds = <span class="number">5</span></span><br><span class="line">    seed = <span class="number">2020</span></span><br><span class="line">    kf = KFold(n_splits=folds, shuffle=<span class="literal">True</span>, random_state=seed)</span><br><span class="line"></span><br><span class="line">    train = np.zeros(train_x.shape[<span class="number">0</span>])</span><br><span class="line">    test = np.zeros(test_x.shape[<span class="number">0</span>])</span><br><span class="line"></span><br><span class="line">    cv_scores = []</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> i, (train_index, valid_index) <span class="keyword">in</span> <span class="built_in">enumerate</span>(kf.split(train_x, train_y)):</span><br><span class="line">        <span class="built_in">print</span>(<span class="string">&#x27;************************************ &#123;&#125; ************************************&#x27;</span>.<span class="built_in">format</span>(<span class="built_in">str</span>(i+<span class="number">1</span>)))</span><br><span class="line">        trn_x, trn_y, val_x, val_y = train_x.iloc[train_index], train_y[train_index], train_x.iloc[valid_index], train_y[valid_index]</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> clf_name == <span class="string">&quot;lgb&quot;</span>:</span><br><span class="line">            train_matrix = clf.Dataset(trn_x, label=trn_y)</span><br><span class="line">            valid_matrix = clf.Dataset(val_x, label=val_y)</span><br><span class="line"></span><br><span class="line">            params = &#123;</span><br><span class="line">                <span class="string">&#x27;boosting_type&#x27;</span>: <span class="string">&#x27;gbdt&#x27;</span>,</span><br><span class="line">                <span class="string">&#x27;objective&#x27;</span>: <span class="string">&#x27;binary&#x27;</span>,</span><br><span class="line">                <span class="string">&#x27;metric&#x27;</span>: <span class="string">&#x27;auc&#x27;</span>,</span><br><span class="line">                <span class="string">&#x27;min_child_weight&#x27;</span>: <span class="number">5</span>,</span><br><span class="line">                <span class="string">&#x27;num_leaves&#x27;</span>: <span class="number">2</span> ** <span class="number">5</span>,</span><br><span class="line">                <span class="string">&#x27;lambda_l2&#x27;</span>: <span class="number">10</span>,</span><br><span class="line">                <span class="string">&#x27;feature_fraction&#x27;</span>: <span class="number">0.8</span>,</span><br><span class="line">                <span class="string">&#x27;bagging_fraction&#x27;</span>: <span class="number">0.8</span>,</span><br><span class="line">                <span class="string">&#x27;bagging_freq&#x27;</span>: <span class="number">4</span>,</span><br><span class="line">                <span class="string">&#x27;learning_rate&#x27;</span>: <span class="number">0.1</span>,</span><br><span class="line">                <span class="string">&#x27;seed&#x27;</span>: <span class="number">2020</span>,</span><br><span class="line">                <span class="string">&#x27;nthread&#x27;</span>: <span class="number">28</span>,</span><br><span class="line">                <span class="string">&#x27;n_jobs&#x27;</span>:<span class="number">24</span>,</span><br><span class="line">                <span class="string">&#x27;silent&#x27;</span>: <span class="literal">True</span>,</span><br><span class="line">                <span class="string">&#x27;verbose&#x27;</span>: -<span class="number">1</span>,</span><br><span class="line">            &#125;</span><br><span class="line"></span><br><span class="line">            model = clf.train(params, train_matrix, <span class="number">50000</span>, valid_sets=[train_matrix, valid_matrix], verbose_eval=<span class="number">200</span>,early_stopping_rounds=<span class="number">200</span>)</span><br><span class="line">            val_pred = model.predict(val_x, num_iteration=model.best_iteration)</span><br><span class="line">            test_pred = model.predict(test_x, num_iteration=model.best_iteration)</span><br><span class="line">            </span><br><span class="line">            <span class="comment"># print(list(sorted(zip(features, model.feature_importance(&quot;gain&quot;)), key=lambda x: x[1], reverse=True))[:20])</span></span><br><span class="line">                </span><br><span class="line">        <span class="keyword">if</span> clf_name == <span class="string">&quot;xgb&quot;</span>:</span><br><span class="line">            train_matrix = clf.DMatrix(trn_x , label=trn_y)</span><br><span class="line">            valid_matrix = clf.DMatrix(val_x , label=val_y)</span><br><span class="line">            </span><br><span class="line">            params = &#123;<span class="string">&#x27;booster&#x27;</span>: <span class="string">&#x27;gbtree&#x27;</span>,</span><br><span class="line">                      <span class="string">&#x27;objective&#x27;</span>: <span class="string">&#x27;binary:logistic&#x27;</span>,</span><br><span class="line">                      <span class="string">&#x27;eval_metric&#x27;</span>: <span class="string">&#x27;auc&#x27;</span>,</span><br><span class="line">                      <span class="string">&#x27;gamma&#x27;</span>: <span class="number">1</span>,</span><br><span class="line">                      <span class="string">&#x27;min_child_weight&#x27;</span>: <span class="number">1.5</span>,</span><br><span class="line">                      <span class="string">&#x27;max_depth&#x27;</span>: <span class="number">5</span>,</span><br><span class="line">                      <span class="string">&#x27;lambda&#x27;</span>: <span class="number">10</span>,</span><br><span class="line">                      <span class="string">&#x27;subsample&#x27;</span>: <span class="number">0.7</span>,</span><br><span class="line">                      <span class="string">&#x27;colsample_bytree&#x27;</span>: <span class="number">0.7</span>,</span><br><span class="line">                      <span class="string">&#x27;colsample_bylevel&#x27;</span>: <span class="number">0.7</span>,</span><br><span class="line">                      <span class="string">&#x27;eta&#x27;</span>: <span class="number">0.04</span>,</span><br><span class="line">                      <span class="string">&#x27;tree_method&#x27;</span>: <span class="string">&#x27;exact&#x27;</span>,</span><br><span class="line">                      <span class="string">&#x27;seed&#x27;</span>: <span class="number">2020</span>,</span><br><span class="line">                      <span class="string">&#x27;nthread&#x27;</span>: <span class="number">36</span>,</span><br><span class="line">                      <span class="string">&quot;silent&quot;</span>: <span class="literal">True</span>,</span><br><span class="line">                      &#125;</span><br><span class="line">            </span><br><span class="line">            watchlist = [(train_matrix, <span class="string">&#x27;train&#x27;</span>),(valid_matrix, <span class="string">&#x27;eval&#x27;</span>)]</span><br><span class="line">            </span><br><span class="line">            model = clf.train(params, train_matrix, num_boost_round=<span class="number">50000</span>, evals=watchlist, verbose_eval=<span class="number">200</span>, early_stopping_rounds=<span class="number">200</span>)</span><br><span class="line">            val_pred  = model.predict(valid_matrix, ntree_limit=model.best_ntree_limit)</span><br><span class="line">            test_pred = model.predict(test_x , ntree_limit=model.best_ntree_limit)</span><br><span class="line">                 </span><br><span class="line">        <span class="keyword">if</span> clf_name == <span class="string">&quot;cat&quot;</span>:</span><br><span class="line">            params = &#123;<span class="string">&#x27;learning_rate&#x27;</span>: <span class="number">0.05</span>, <span class="string">&#x27;depth&#x27;</span>: <span class="number">5</span>, <span class="string">&#x27;l2_leaf_reg&#x27;</span>: <span class="number">10</span>, <span class="string">&#x27;bootstrap_type&#x27;</span>: <span class="string">&#x27;Bernoulli&#x27;</span>,</span><br><span class="line">                      <span class="string">&#x27;od_type&#x27;</span>: <span class="string">&#x27;Iter&#x27;</span>, <span class="string">&#x27;od_wait&#x27;</span>: <span class="number">50</span>, <span class="string">&#x27;random_seed&#x27;</span>: <span class="number">11</span>, <span class="string">&#x27;allow_writing_files&#x27;</span>: <span class="literal">False</span>&#125;</span><br><span class="line">            </span><br><span class="line">            model = clf(iterations=<span class="number">20000</span>, **params)</span><br><span class="line">            model.fit(trn_x, trn_y, eval_set=(val_x, val_y),</span><br><span class="line">                      cat_features=[], use_best_model=<span class="literal">True</span>, verbose=<span class="number">500</span>)</span><br><span class="line">            </span><br><span class="line">            val_pred  = model.predict(val_x)</span><br><span class="line">            test_pred = model.predict(test_x)</span><br><span class="line">            </span><br><span class="line">        train[valid_index] = val_pred</span><br><span class="line">        test = test_pred / kf.n_splits</span><br><span class="line">        cv_scores.append(roc_auc_score(val_y, val_pred))</span><br><span class="line">        </span><br><span class="line">        <span class="built_in">print</span>(cv_scores)</span><br><span class="line">        </span><br><span class="line">    <span class="built_in">print</span>(<span class="string">&quot;%s_scotrainre_list:&quot;</span> % clf_name, cv_scores)</span><br><span class="line">    <span class="built_in">print</span>(<span class="string">&quot;%s_score_mean:&quot;</span> % clf_name, np.mean(cv_scores))</span><br><span class="line">    <span class="built_in">print</span>(<span class="string">&quot;%s_score_std:&quot;</span> % clf_name, np.std(cv_scores))</span><br><span class="line">    <span class="keyword">return</span> train, test</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">lgb_model</span>(<span class="params">x_train, y_train, x_test</span>):</span></span><br><span class="line">    lgb_train, lgb_test = cv_model(lgb, x_train, y_train, x_test, <span class="string">&quot;lgb&quot;</span>)</span><br><span class="line">    <span class="keyword">return</span> lgb_train, lgb_test</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">xgb_model</span>(<span class="params">x_train, y_train, x_test</span>):</span></span><br><span class="line">    xgb_train, xgb_test = cv_model(xgb, x_train, y_train, x_test, <span class="string">&quot;xgb&quot;</span>)</span><br><span class="line">    <span class="keyword">return</span> xgb_train, xgb_test</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">cat_model</span>(<span class="params">x_train, y_train, x_test</span>):</span></span><br><span class="line">    cat_train, cat_test = cv_model(CatBoostRegressor, x_train, y_train, x_test, <span class="string">&quot;cat&quot;</span>)</span><br></pre></td></tr></table></figure>


<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">lgb_train, lgb_test = lgb_model(x_train, y_train, x_test)</span><br></pre></td></tr></table></figure>

<pre><code>************************************ 1 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
[LightGBM] [Warning] Unknown parameter: silent
Training until validation scores don&#39;t improve for 200 rounds
[200]    training&#39;s auc: 0.749114    valid_1&#39;s auc: 0.729275
[400]    training&#39;s auc: 0.764716    valid_1&#39;s auc: 0.730125
[600]    training&#39;s auc: 0.778489    valid_1&#39;s auc: 0.729928
Early stopping, best iteration is:
[446]    training&#39;s auc: 0.768137    valid_1&#39;s auc: 0.730186
[0.7301862239949224]
************************************ 2 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
[LightGBM] [Warning] Unknown parameter: silent
Training until validation scores don&#39;t improve for 200 rounds
[200]    training&#39;s auc: 0.748999    valid_1&#39;s auc: 0.731035
[400]    training&#39;s auc: 0.764879    valid_1&#39;s auc: 0.731436
[600]    training&#39;s auc: 0.778506    valid_1&#39;s auc: 0.730823
Early stopping, best iteration is:
[414]    training&#39;s auc: 0.765823    valid_1&#39;s auc: 0.731478
[0.7301862239949224, 0.7314779648434573]
************************************ 3 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
[LightGBM] [Warning] Unknown parameter: silent
Training until validation scores don&#39;t improve for 200 rounds
[200]    training&#39;s auc: 0.748145    valid_1&#39;s auc: 0.73253
[400]    training&#39;s auc: 0.763814    valid_1&#39;s auc: 0.733272
[600]    training&#39;s auc: 0.777895    valid_1&#39;s auc: 0.733354
Early stopping, best iteration is:
[475]    training&#39;s auc: 0.769215    valid_1&#39;s auc: 0.73355
[0.7301862239949224, 0.7314779648434573, 0.7335502065719879]
************************************ 4 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
[LightGBM] [Warning] Unknown parameter: silent
Training until validation scores don&#39;t improve for 200 rounds
[200]    training&#39;s auc: 0.749417    valid_1&#39;s auc: 0.727507
[400]    training&#39;s auc: 0.765066    valid_1&#39;s auc: 0.728261
Early stopping, best iteration is:
[353]    training&#39;s auc: 0.761647    valid_1&#39;s auc: 0.728349
[0.7301862239949224, 0.7314779648434573, 0.7335502065719879, 0.7283491938614568]
************************************ 5 ************************************
[LightGBM] [Warning] num_threads is set with nthread=28, will be overridden by n_jobs=24. Current value: num_threads=24
[LightGBM] [Warning] Unknown parameter: silent
Training until validation scores don&#39;t improve for 200 rounds
[200]    training&#39;s auc: 0.748562    valid_1&#39;s auc: 0.73262
[400]    training&#39;s auc: 0.764493    valid_1&#39;s auc: 0.733365
Early stopping, best iteration is:
[394]    training&#39;s auc: 0.764109    valid_1&#39;s auc: 0.733381
[0.7301862239949224, 0.7314779648434573, 0.7335502065719879, 0.7283491938614568, 0.7333810157041901]
lgb_scotrainre_list: [0.7301862239949224, 0.7314779648434573, 0.7335502065719879, 0.7283491938614568, 0.7333810157041901]
lgb_score_mean: 0.7313889209952029
lgb_score_std: 0.001966415347937543
</code></pre>
<p><img src= "/a_zhangqingxiang.gitee.io/img/loading.gif" data-lazy-src="https://img-blog.csdnimg.cn/20200921220620291.png#pic_center" alt="在这里插入图片描述"></p>
<h4 id="总结"><a href="#总结" class="headerlink" title="总结"></a>总结</h4><p>最后在此声明特征工程是数据挖掘中最为重要的一部分，在往往比赛中，还是符合八二原则。占据着最为重要一部分。一定要好好学习</p>
<p><strong>最后阐述一下特征选择</strong></p>
<h4 id="特征选择"><a href="#特征选择" class="headerlink" title="特征选择"></a>特征选择</h4><p>特征选择技术可以精简掉无用的特征，以降低最终模型的复杂性，它的最终目的是得到一个简约模型，在不降低预测准确率或对预测准确率影响不大的情况下提高计算速度。特征选择不是为了减少训练时间（实际上，一些技术会增加总体训练时间），而是为了减少模型评分时间。<br>特征选择的方法：</p>
<p><strong>1、 Filter</strong></p>
<ul>
<li>方差选择法 ：方差选择法中，先要计算各个特征的方差，然后根据设定的阈值，选择方差大于阈值的特征</li>
<li>相关系数法（pearson 相关系数）：Pearson 相关系数 皮尔森相关系数是一种最简单的，可以帮助理解特征和响应变量之间关系的方法，该方法衡量的是变量之间的线性相关性。 结果的取值区间为 [-1，1] ， -1 表示完全的负相关， +1表示完全的正相关，0 表示没有线性相关。</li>
<li>卡方检验：经典的卡方检验是用于检验自变量对因变量的相关性。 假设自变量有N种取值，因变量有M种取值，考虑自变量等于i且因变量等于j的样本频数的观察值与期望的差距。 其统计量如下： χ2=∑(A−T)2T，其中A为实际值，T为理论值<br>(注：卡方只能运用在正定矩阵上，否则会报错Input X must be non-negative)</li>
<li>互信息法： 经典的互信息也是评价自变量对因变量的相关性的。 在feature_selection库的SelectKBest类结合最大信息系数法可以用于选择特征</li>
</ul>
<p><strong>2、Wrapper （RFE）</strong></p>
<ul>
<li>递归特征消除法：递归特征消除法 递归消除特征法使用一个基模型来进行多轮训练，每轮训练后，消除若干权值系数的特征，再基于新的特征集进行下一轮训练。</li>
</ul>
<p><strong>3、Embedded</strong> </p>
<ul>
<li>基于惩罚项的特征选择法： 基于惩罚项的特征选择法 使用带惩罚项的基模型，除了筛选出特征外，同时也进行了降维。在feature_selection库的SelectFromModel类结合逻辑回归模型可以用于选择特征</li>
<li>基于树模型的特征选择：基于树模型的特征选择 树模型中GBDT也可用来作为基模型进行特征选择。 在feature_selection库的SelectFromModel类结合GBDT模型可以用于选择特征</li>
</ul>
<h4 id="结束语"><a href="#结束语" class="headerlink" title="结束语"></a>结束语</h4><p>本次任务三学习到不少知识，文章中如有不足之处，请务必指出，一定迅速改正。谢谢</p>
</article><div class="post-copyright"><div class="post-copyright__author"><span class="post-copyright-meta">文章作者: </span><span class="post-copyright-info"><a href="mailto:344285081@qq.com">cattle_coder</a></span></div><div class="post-copyright__type"><span class="post-copyright-meta">文章链接: </span><span class="post-copyright-info"><a href="https://azhangqingxiang.gitee.io/a_zhangqingxiang.gitee.io/2021/05/03/%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8BTask3%EF%BC%9A%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B/">https://azhangqingxiang.gitee.io/a_zhangqingxiang.gitee.io/2021/05/03/%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8BTask3%EF%BC%9A%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B/</a></span></div><div class="post-copyright__notice"><span class="post-copyright-meta">版权声明: </span><span class="post-copyright-info">本博客所有文章除特别声明外，均采用 <a href="https://creativecommons.org/licenses/by-nc-sa/4.0/" target="_blank">CC BY-NC-SA 4.0</a> 许可协议。转载请注明来自 <a href="https://azhangqingxiang.gitee.io/a_zhangqingxiang.gitee.io" target="_blank">cattle_coder</a>！</span></div></div><div class="tag_share"><div class="post-meta__tag-list"></div><div class="post_share"><div class="addthis_inline_share_toolbox"></div><script src="//s7.addthis.com/js/300/addthis_widget.js#pubid=https://www.addthis.com/" async="async"></script></div></div><div class="post-reward"><div class="reward-button button--animated"><i class="fas fa-qrcode"></i> 打赏</div><div class="reward-main"><ul class="reward-all"><li class="reward-item"><a href="/a_zhangqingxiang.gitee.io/img/wechat.jpg" target="_blank"><img class="post-qr-code-img" data-lazy-src="/a_zhangqingxiang.gitee.io/img/wechat.jpg" alt="微信"/></a><div class="post-qr-code-desc">微信</div></li><li class="reward-item"><a href="/a_zhangqingxiang.gitee.io/img/alipay.jpg" target="_blank"><img class="post-qr-code-img" data-lazy-src="/a_zhangqingxiang.gitee.io/img/alipay.jpg" alt="支付宝"/></a><div class="post-qr-code-desc">支付宝</div></li></ul></div></div><nav class="pagination-post" id="pagination"><div class="prev-post pull-left"><a href="/a_zhangqingxiang.gitee.io/2021/05/03/%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8BTask4%EF%BC%9A%E5%BB%BA%E6%A8%A1%E5%92%8C%E8%B0%83%E5%8F%82/"><img class="prev-cover" data-lazy-src="https://ftp.bmp.ovh/imgs/2021/05/9f923c2fad4291e9.jpg" onerror="onerror=null;src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="cover of previous post"><div class="pagination-info"><div class="label">上一篇</div><div class="prev_info">零基础入门金融风控之贷款违约预测Task4：建模和调参</div></div></a></div><div class="next-post pull-right"><a href="/a_zhangqingxiang.gitee.io/2021/05/01/%E5%89%8D%E7%A8%8B%E6%97%A0%E5%BF%A7%E6%8B%9B%E8%81%98%E4%BF%A1%E6%81%AF%E7%88%AC%E5%8F%96/"><img class="next-cover" data-lazy-src="https://i.loli.net/2021/05/05/TI9CzyGpiSoOAfm.jpg" onerror="onerror=null;src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="cover of next post"><div class="pagination-info"><div class="label">下一篇</div><div class="next_info">爬取前程无忧招聘信息</div></div></a></div></nav></div><div class="aside-content" id="aside-content"><div class="card-widget card-info"><div class="card-info-avatar is-center"><img class="avatar-img" data-lazy-src="https://i.loli.net/2021/05/03/fjVLMHmWeun5wxX.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/friend_404.gif'" alt="avatar"/><div class="author-info__name">cattle_coder</div><div class="author-info__description">学习使我快乐</div></div><div class="card-info-data"><div class="card-info-data-item is-center"><a href="/a_zhangqingxiang.gitee.io/archives/"><div class="headline">文章</div><div class="length-num">15</div></a></div><div class="card-info-data-item is-center"><a href="/a_zhangqingxiang.gitee.io/tags/"><div class="headline">标签</div><div class="length-num">2</div></a></div><div class="card-info-data-item is-center"><a href="/a_zhangqingxiang.gitee.io/categories/"><div class="headline">分类</div><div class="length-num">6</div></a></div></div><a class="button--animated" id="card-info-btn" target="_blank" rel="noopener" href="https://github.com/Data-zhang-0"><i class="fab fa-github"></i><span>Follow Me</span></a></div><div class="card-widget card-announcement"><div class="item-headline"><i class="fas fa-bullhorn card-announcement-animation"></i><span>公告</span></div><div class="announcement_content">醒醒该干饭了</div></div><div class="sticky_layout"><div class="card-widget" id="card-toc"><div class="item-headline"><i class="fas fa-stream"></i><span>目录</span></div><div class="toc-content"><ol class="toc"><li class="toc-item toc-level-1"><a class="toc-link" href="#%E9%9B%B6%E5%9F%BA%E7%A1%80%E5%85%A5%E9%97%A8%E9%87%91%E8%9E%8D%E9%A3%8E%E6%8E%A7%E4%B9%8B%E8%B4%B7%E6%AC%BE%E8%BF%9D%E7%BA%A6%E9%A2%84%E6%B5%8B"><span class="toc-number">1.</span> <span class="toc-text">零基础入门金融风控之贷款违约预测</span></a></li><li class="toc-item toc-level-1"><a class="toc-link" href="#Task3%EF%BC%9A%E7%89%B9%E5%BE%81%E5%B7%A5%E7%A8%8B"><span class="toc-number">2.</span> <span class="toc-text">Task3：特征工程</span></a><ol class="toc-child"><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%89%B9%E5%BE%81%E9%A2%84%E5%A4%84%E7%90%86"><span class="toc-number">2.0.0.1.</span> <span class="toc-text">特征预处理</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%BC%BA%E5%A4%B1%E5%80%BC%E5%A4%84%E7%90%86"><span class="toc-number">2.0.0.2.</span> <span class="toc-text">缺失值处理</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E6%97%B6%E9%97%B4%E6%A0%BC%E5%BC%8F%E5%A4%84%E7%90%86"><span class="toc-number">2.0.0.3.</span> <span class="toc-text">时间格式处理</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E5%AF%B9%E8%B1%A1%E7%89%B9%E5%BE%81%E8%BD%AC%E5%8C%96%E6%95%B0%E5%80%BC%E7%89%B9%E5%BE%81"><span class="toc-number">2.0.0.4.</span> <span class="toc-text">对象特征转化数值特征</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%B1%BB%E5%88%AB%E7%89%B9%E5%BE%81%E5%A4%84%E7%90%86"><span class="toc-number">2.0.0.5.</span> <span class="toc-text">类别特征处理</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E5%BC%82%E5%B8%B8%E5%80%BC%E5%A4%84%E7%90%86"><span class="toc-number">2.0.0.6.</span> <span class="toc-text">异常值处理</span></a><ol class="toc-child"><li class="toc-item toc-level-5"><a class="toc-link" href="#%E5%AF%B9%E4%BA%8E%E6%A3%80%E6%9F%A5%E5%BC%82%E5%B8%B8%E6%96%B9%E6%B3%95%E4%B9%8B%E4%B8%80%EF%BC%9A%E5%9D%87%E6%96%B9%E5%B7%AE"><span class="toc-number">2.0.0.6.1.</span> <span class="toc-text">对于检查异常方法之一：均方差</span></a></li><li class="toc-item toc-level-5"><a class="toc-link" href="#%E6%96%B9%E6%B3%95%E4%BA%8C%EF%BC%9A%E7%AE%B1%E5%9E%8B%E5%9B%BE"><span class="toc-number">2.0.0.6.2.</span> <span class="toc-text">方法二：箱型图</span></a></li></ol></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%89%B9%E5%BE%81%E4%BA%A4%E4%BA%92"><span class="toc-number">2.0.0.7.</span> <span class="toc-text">特征交互</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E6%80%BB%E7%BB%93"><span class="toc-number">2.0.0.8.</span> <span class="toc-text">总结</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%89%B9%E5%BE%81%E9%80%89%E6%8B%A9"><span class="toc-number">2.0.0.9.</span> <span class="toc-text">特征选择</span></a></li><li class="toc-item toc-level-4"><a class="toc-link" href="#%E7%BB%93%E6%9D%9F%E8%AF%AD"><span class="toc-number">2.0.0.10.</span> <span class="toc-text">结束语</span></a></li></ol></li></ol></li></ol></li></ol></div></div><div class="card-widget card-recent-post"><div class="item-headline"><i class="fas fa-history"></i><span>最新文章</span></div><div class="aside-list"><div class="aside-list-item"><a class="thumbnail" href="/a_zhangqingxiang.gitee.io/2021/05/03/GBDT+LR%E7%AE%80%E4%BB%8B/" title="一. GBDT+LR简介"><img data-lazy-src="https://i.loli.net/2019/12/25/8t97aVlp4hgyBGu.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="一. GBDT+LR简介"/></a><div class="content"><a class="title" href="/a_zhangqingxiang.gitee.io/2021/05/03/GBDT+LR%E7%AE%80%E4%BB%8B/" title="一. GBDT+LR简介">一. GBDT+LR简介</a><time datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time></div></div><div class="aside-list-item"><a class="thumbnail" href="/a_zhangqingxiang.gitee.io/2021/05/03/Mindspore%E7%9A%84%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B/" title="Mindspore快速上手"><img data-lazy-src="https://i.loli.net/2021/05/05/advGj3UrV5CYlbc.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="Mindspore快速上手"/></a><div class="content"><a class="title" href="/a_zhangqingxiang.gitee.io/2021/05/03/Mindspore%E7%9A%84%E5%BF%AB%E9%80%9F%E4%B8%8A%E6%89%8B/" title="Mindspore快速上手">Mindspore快速上手</a><time datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time></div></div><div class="aside-list-item"><a class="thumbnail" href="/a_zhangqingxiang.gitee.io/2021/05/03/Wide%E5%92%8CDeep%E6%A8%A1%E5%9E%8B/" title="Wide和Deep模型"><img data-lazy-src="https://i.loli.net/2021/05/05/Utv7QgBwWfKL8XV.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="Wide和Deep模型"/></a><div class="content"><a class="title" href="/a_zhangqingxiang.gitee.io/2021/05/03/Wide%E5%92%8CDeep%E6%A8%A1%E5%9E%8B/" title="Wide和Deep模型">Wide和Deep模型</a><time datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time></div></div><div class="aside-list-item"><a class="thumbnail" href="/a_zhangqingxiang.gitee.io/2021/05/03/python%E5%BC%80%E5%8F%91%E5%85%88%E5%88%9B%E5%BB%BA%E8%99%9A%E6%8B%9F%E7%8E%AF%E5%A2%83%E5%91%80/" title="python开发先创建虚拟环境呀"><img data-lazy-src="https://i.loli.net/2021/05/05/x3MJV49qNSnjhwe.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="python开发先创建虚拟环境呀"/></a><div class="content"><a class="title" href="/a_zhangqingxiang.gitee.io/2021/05/03/python%E5%BC%80%E5%8F%91%E5%85%88%E5%88%9B%E5%BB%BA%E8%99%9A%E6%8B%9F%E7%8E%AF%E5%A2%83%E5%91%80/" title="python开发先创建虚拟环境呀">python开发先创建虚拟环境呀</a><time datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time></div></div><div class="aside-list-item"><a class="thumbnail" href="/a_zhangqingxiang.gitee.io/2021/05/03/%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9Ftask01%EF%BC%9A%E7%AE%80%E4%BB%8B%E6%A6%82%E8%BF%B0/" title="推荐系统task01：简介概述"><img data-lazy-src="https://i.loli.net/2021/05/05/ZFInOwm1QGdP2La.jpg" onerror="this.onerror=null;this.src='/a_zhangqingxiang.gitee.io/img/404.jpg'" alt="推荐系统task01：简介概述"/></a><div class="content"><a class="title" href="/a_zhangqingxiang.gitee.io/2021/05/03/%E6%8E%A8%E8%8D%90%E7%B3%BB%E7%BB%9Ftask01%EF%BC%9A%E7%AE%80%E4%BB%8B%E6%A6%82%E8%BF%B0/" title="推荐系统task01：简介概述">推荐系统task01：简介概述</a><time datetime="2021-05-03T08:42:09.000Z" title="发表于 2021-05-03 16:42:09">2021-05-03</time></div></div></div></div></div></div></main><footer id="footer" style="background-image: url('https://i.loli.net/2021/05/05/ZMyANcLThdbtsiC.jpg')"><div id="footer-wrap"><div class="copyright">&copy;2021 By cattle_coder</div><div class="framework-info"><span>框架 </span><a target="_blank" rel="noopener" href="https://hexo.io">Hexo</a><span class="footer-separator">|</span><span>主题 </span><a target="_blank" rel="noopener" href="https://github.com/jerryc127/hexo-theme-butterfly">Butterfly</a></div></div></footer></div><div id="rightside"><div id="rightside-config-hide"><button id="readmode" type="button" title="阅读模式"><i class="fas fa-book-open"></i></button><button id="darkmode" type="button" title="浅色和深色模式转换"><i class="fas fa-adjust"></i></button><button id="hide-aside-btn" type="button" title="单栏和双栏切换"><i class="fas fa-arrows-alt-h"></i></button></div><div id="rightside-config-show"><button id="rightside_config" type="button" title="设置"><i class="fas fa-cog fa-spin"></i></button><button class="close" id="mobile-toc-button" type="button" title="目录"><i class="fas fa-list-ul"></i></button><button id="go-up" type="button" title="回到顶部"><i class="fas fa-arrow-up"></i></button></div></div><div><script src="/a_zhangqingxiang.gitee.io/js/utils.js"></script><script src="/a_zhangqingxiang.gitee.io/js/main.js"></script><script src="https://cdn.jsdelivr.net/npm/vanilla-lazyload/dist/lazyload.iife.min.js"></script><div class="js-pjax"></div><script id="canvas_nest" defer="defer" color="0,0,255" opacity="0.9" zIndex="-1" count="99" mobile="false" src="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/dist/canvas-nest.min.js"></script><script id="click-heart" src="https://cdn.jsdelivr.net/npm/butterfly-extsrc@1/dist/click-heart.min.js" async="async" mobile="false"></script><script async data-pjax src="//busuanzi.ibruce.info/busuanzi/2.3/busuanzi.pure.mini.js"></script></div></body></html>